{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 推荐系统 & 点击率预估\n",
    "\n",
    "Event Recommendation Engine Challenge 是Kaggle 上的一个推荐任务竞赛，根据用户的历史参加活动记录、用户的社交信息、以及用户历史上在App上浏览和点击的活动，预测用户是否会对某个活动感兴趣。  \n",
    "\n",
    "以下代码实现内容：\n",
    "1. 实现基于用户的协同过滤（用户本身，用户好友分析）  \n",
    "2. 实现基于活动的协同过滤（活动本身，用户活动关联分析）  \n",
    "3. 能够实现基于模型的协同过滤（矩阵分解/LFM）  \n",
    "4. 组合各种前面生成的特征，生成新的训练数据，并对数据进行保存  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from __future__ import division\n",
    "\n",
    "import pickle\n",
    "import numpy as np\n",
    "import scipy.io as sio\n",
    "import scipy.sparse as ss\n",
    "from numpy.random import random  \n",
    "from collections import defaultdict"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "导入必要工具包（精确除法、稀疏矩阵处理等）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'\\nuserIndex = pickle.load(open(\"PE_userIndex.pkl\", \\'rb\\'))\\neventIndex = pickle.load(open(\"PE_eventIndex.pkl\", \\'rb\\'))\\nn_users = len(userIndex)\\nn_events = len(eventIndex)\\n\\n# 建立存储用户相似度的矩阵，对角线元素为1.0\\nuserSimMatrix_userBased= ss.dok_matrix((n_users, n_users))\\nfor i in range(n_users):\\n    userSimMatrix_userBased[i,i] = 1.0\\n    \\n# 导入关联用户对\\nuniqueUserPairs = pickle.load(open(\"FE_uniqueUserPairs.pkl\", \\'rb\\'))\\n# 每个用户参加的活动\\neventsForUser = pickle.load(open(\"PE_eventsForUser.pkl\", \\'rb\\'))\\n# 取出关联用户对参加活动的交集（共同给哪个活动打分了）\\nfor u1, u2 in uniqueUserPairs:\\n    i = u1\\n    j = u2\\n    set1 = eventsForUser[u1]\\n    set2 = eventsForUser[u2]\\n    set3 = set1 & set2\\n    print(set3)\\n    #print(\"******************\")\\n    #print(list2)\\n'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"\"\"\n",
    "userIndex = pickle.load(open(\"PE_userIndex.pkl\", 'rb'))\n",
    "eventIndex = pickle.load(open(\"PE_eventIndex.pkl\", 'rb'))\n",
    "n_users = len(userIndex)\n",
    "n_events = len(eventIndex)\n",
    "\n",
    "# 建立存储用户相似度的矩阵，对角线元素为1.0\n",
    "userSimMatrix_userBased= ss.dok_matrix((n_users, n_users))\n",
    "for i in range(n_users):\n",
    "    userSimMatrix_userBased[i,i] = 1.0\n",
    "    \n",
    "# 导入关联用户对\n",
    "uniqueUserPairs = pickle.load(open(\"FE_uniqueUserPairs.pkl\", 'rb'))\n",
    "# 每个用户参加的活动\n",
    "eventsForUser = pickle.load(open(\"PE_eventsForUser.pkl\", 'rb'))\n",
    "# 取出关联用户对参加活动的交集（共同给哪个活动打分了）\n",
    "for u1, u2 in uniqueUserPairs:\n",
    "    i = u1\n",
    "    j = u2\n",
    "    set1 = eventsForUser[u1]\n",
    "    set2 = eventsForUser[u2]\n",
    "    set3 = set1 & set2\n",
    "    print(set3)\n",
    "    #print(\"******************\")\n",
    "    #print(list2)\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在计算用户之间相似度时，本来想选择关联用户，然后用他们共同参加过的events（都打过分）计算相似度，而不是取用户对应的userEventScore里面一整行的打分来计算相似度，因为整行里面（每个user对应13428个events）可能有很多活动用户没有参加，我感觉只选择两个用户都参加过的活动计算相似度更合理一点，但是发现取出来关联用户参加的events交集有很多只有一个，这样代入公式没法计算，所以还是直接使用整行，对于没有参加过的活动在计算相似度时可能会得到零值，所以公式计算时可能会忽略这些不关联的用户  \n",
    "相似度的计算使用了评分矩阵的整行数据，但在实际计算预测活动推荐度时，使用的是和目标用户相关联的用户作为参考并协同推荐"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 计算相似度\n",
    "用户之间相似度：首先需要用数学的方法表示用户，即用向量，向量的内容则是该用户对不同活动的评价，使用不同用户的向量来计算相似度  \n",
    "活动之间相似度：首先需要用数学的方法表示活动，即用向量，向量的内容则是不同用户对该活动评价，使用不同活动的向量来计算相似度  \n",
    "\n",
    "相似度取值：[-1,1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 计算用户之间的相似度矩阵，便于后续直接查询"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下方计算基于用户的协同过滤时，需要频繁使用到用户之间的相似度，为了减少计算量，可以提前将所有用户的相似度计算好并存储在矩阵中，后续直接在矩阵中查询取值就行"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0. 0. 0. ... 0. 0. 0.]\n",
      " [0. 0. 0. ... 0. 0. 0.]\n",
      " [0. 0. 0. ... 0. 0. 0.]\n",
      " ...\n",
      " [0. 0. 0. ... 0. 0. 0.]\n",
      " [0. 0. 0. ... 0. 0. 0.]\n",
      " [0. 0. 0. ... 0. 0. 0.]]\n",
      "<class 'numpy.matrixlib.defmatrix.matrix'>\n",
      "[[0.00000000e+00]\n",
      " [0.00000000e+00]\n",
      " [1.49053510e-04]\n",
      " ...\n",
      " [0.00000000e+00]\n",
      " [0.00000000e+00]\n",
      " [7.45267551e-05]]\n",
      "(3391, 1)\n"
     ]
    }
   ],
   "source": [
    "# 导入用户和活动的重编码表（前期已进行处理，只取训练集和测试集出现过的用户和活动）\n",
    "userIndex = pickle.load(open(\"PE_userIndex.pkl\", 'rb'))\n",
    "eventIndex = pickle.load(open(\"PE_eventIndex.pkl\", 'rb'))\n",
    "n_users = len(userIndex)\n",
    "n_events = len(eventIndex)\n",
    "\n",
    "# 导入关联用户对和分数矩阵\n",
    "uniqueUserPairs = pickle.load(open(\"FE_uniqueUserPairs.pkl\", 'rb'))\n",
    "userEventScores = sio.mmread(\"PE_userEventScores\").todense()\n",
    "print(userEventScores)\n",
    "print(type(userEventScores))                  # 验证评分矩阵为稀疏矩阵 [3391, 13418]\n",
    "userEventScores = np.mat(userEventScores)     # np.mat作用是转化为np中定义的稀疏矩阵，可能不影响，但总觉得统一格式好点\n",
    "#print(userEventScores)\n",
    "#print(type(userEventScores))                 # 打印结果和未转化前一致\n",
    "\n",
    "mean_score = np.mean(userEventScores,axis=1)  # 相似度矩阵中需要计算每个user打分的均值，提前计算并得到矩阵形式 [1，13428]\n",
    "print(mean_score)\n",
    "print(mean_score.shape)                       # 得到 [3391,1] 维数的均值矩阵\n",
    "\n",
    "row_userEventScores = userEventScores - mean_score    # 计算每行的偏差（单个event的分数减均值）\n",
    "row_userEventScores = np.array(row_userEventScores)\n",
    "# 不转化为array，下面相乘时提示元素没有对齐，可能矩阵相乘时我没有转置，两个都是行矩阵\n",
    "# 转换为array就不会提示，直接对应相乘\n",
    "\n",
    "\n",
    "# 建立存储用户相似度的矩阵，对角线元素为1.0\n",
    "userSimMatrix_userBased= ss.dok_matrix((n_users, n_users))\n",
    "for i in range(n_users):\n",
    "    userSimMatrix_userBased[i,i] = 1.0\n",
    "    \n",
    "    \n",
    "# 取出关联用户对应的活动分数，进行相似度计算，公式中直接用矩阵和向量的方式计算，比轮询计算单个数值快\n",
    "# 此处计算相似度使用 Pearson相关系数公式，公式形式有多种，前面代码计算基本用户本身属性的用户间相似度使用的是另一种形式\n",
    "for u1, u2 in uniqueUserPairs:\n",
    "    i = u1\n",
    "    j = u2\n",
    "    if(i,j) not in userSimMatrix_userBased.keys():\n",
    "        sim = sum(row_userEventScores[i]*row_userEventScores[j])\\\n",
    "             /(np.sqrt(sum(np.power(row_userEventScores[i],2))))\\\n",
    "             /(np.sqrt(sum(np.power(row_userEventScores[j],2))))\n",
    "        #print(sim)\n",
    "        userSimMatrix_userBased[i,j] = sim\n",
    "        userSimMatrix_userBased[j,i] = sim\n",
    "sio.mmwrite('userSimMatrix_userBased',userSimMatrix_userBased)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[-0.00037263 -0.00037263 -0.00037263 ... -0.00037263 -0.00037263\n",
      " -0.00037263]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(13418,)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(row_userEventScores[i])\n",
    "row_userEventScores[i].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3391, 13418)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "userEventScores.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  (0, 0)\t1.0\n",
      "  (1, 1)\t1.0\n",
      "  (2, 2)\t1.0\n",
      "  (3, 3)\t1.0\n",
      "  (4, 4)\t1.0\n",
      "  (5, 5)\t1.0\n",
      "  (6, 6)\t1.0\n",
      "  (7, 7)\t1.0\n",
      "  (8, 8)\t1.0\n",
      "  (9, 9)\t1.0\n",
      "  (10, 10)\t1.0\n",
      "  (11, 11)\t1.0\n",
      "  (12, 12)\t1.0\n",
      "  (13, 13)\t1.0\n",
      "  (14, 14)\t1.0\n",
      "  (15, 15)\t1.0\n",
      "  (16, 16)\t1.0\n",
      "  (17, 17)\t1.0\n",
      "  (18, 18)\t1.0\n",
      "  (19, 19)\t1.0\n",
      "  (20, 20)\t1.0\n",
      "  (21, 21)\t1.0\n",
      "  (22, 22)\t1.0\n",
      "  (23, 23)\t1.0\n",
      "  (24, 24)\t1.0\n",
      "  :\t:\n",
      "  (2908, 3274)\t-7.45323097563392e-05\n",
      "  (1869, 1515)\t0.6545560631824034\n",
      "  (1515, 1869)\t0.6545560631824034\n",
      "  (1051, 1280)\t-0.00010540853153624303\n",
      "  (1280, 1051)\t-0.00010540853153624303\n",
      "  (3083, 1610)\t-0.0002582356120275864\n",
      "  (1610, 3083)\t-0.0002582356120275864\n",
      "  (1235, 435)\t-0.0001825865412756965\n",
      "  (435, 1235)\t-0.0001825865412756965\n",
      "  (1092, 360)\t-0.00012910337003435858\n",
      "  (360, 1092)\t-0.00012910337003435858\n",
      "  (317, 1130)\t0.4999440976427195\n",
      "  (1130, 317)\t0.4999440976427195\n",
      "  (3081, 2697)\t0.5773072363368542\n",
      "  (2697, 3081)\t0.5773072363368542\n",
      "  (211, 987)\t-0.00010540853153622814\n",
      "  (987, 211)\t-0.00010540853153622814\n",
      "  (1678, 3386)\t-0.0003497309043620044\n",
      "  (3386, 1678)\t-0.0003497309043620044\n",
      "  (2110, 2911)\t-0.00010540853153624288\n",
      "  (2911, 2110)\t-0.00010540853153624288\n",
      "  (2732, 1291)\t-0.0001666841596981052\n",
      "  (1291, 2732)\t-0.0001666841596981052\n",
      "  (1825, 1450)\t-0.00023573578440772513\n",
      "  (1450, 1825)\t-0.00023573578440772513\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(3391, 3391)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(userSimMatrix_userBased)\n",
    "userSimMatrix_userBased.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看出得出的相似度矩阵也很稀疏，而相似系数也比较合理，其中出现负值可能是因为两个用户之间的行为刚好相反或很不一样"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 计算活动之间的相似度矩阵，便于后续直接查询"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "活动之间的相似度计算和用户相似度计算有点类似，上面的一些数据可以直接使用（如用户打分相对均值的偏差等）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 导入关联活动对\n",
    "uniqueEventPairs = pickle.load(open(\"PE_uniqueEventPairs.pkl\", 'rb'))\n",
    "\n",
    "# 建立存储活动相似度的矩阵，对角线元素为1.0\n",
    "eventSimMatrix_eventBased = ss.dok_matrix((n_events, n_events))\n",
    "for i in range(n_events):\n",
    "    eventSimMatrix_eventBased[i,i] = 1.0\n",
    "\n",
    "# 取出关联活动对应的分数，进行相似度计算   维数 [13418， 13418]\n",
    "# 使用夹角余弦公式计算\n",
    "for event1, event2 in uniqueUserPairs:\n",
    "    m = event1\n",
    "    n = event2\n",
    "    if(m,n) not in eventSimMatrix_eventBased.keys():\n",
    "        sim_e = sum(row_userEventScores[:,m]*row_userEventScores[:,n])\\\n",
    "             /(np.sqrt(sum(np.power(row_userEventScores[:,m],2))))\\\n",
    "             /(np.sqrt(sum(np.power(row_userEventScores[:,n],2))))\n",
    "        #print(sim)\n",
    "        eventSimMatrix_eventBased[m,n] = sim_e\n",
    "        eventSimMatrix_eventBased[n,m] = sim_e\n",
    "sio.mmwrite('eventSimMatrix_eventBased',eventSimMatrix_eventBased)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.00000000e+00  0.00000000e+00 -1.49053510e-04 ...  0.00000000e+00\n",
      "  0.00000000e+00 -7.45267551e-05]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(3391,)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(row_userEventScores[:,m])\n",
    "row_userEventScores[:,m].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  (0, 0)\t1.0\n",
      "  (1, 1)\t1.0\n",
      "  (2, 2)\t1.0\n",
      "  (3, 3)\t1.0\n",
      "  (4, 4)\t1.0\n",
      "  (5, 5)\t1.0\n",
      "  (6, 6)\t1.0\n",
      "  (7, 7)\t1.0\n",
      "  (8, 8)\t1.0\n",
      "  (9, 9)\t1.0\n",
      "  (10, 10)\t1.0\n",
      "  (11, 11)\t1.0\n",
      "  (12, 12)\t1.0\n",
      "  (13, 13)\t1.0\n",
      "  (14, 14)\t1.0\n",
      "  (15, 15)\t1.0\n",
      "  (16, 16)\t1.0\n",
      "  (17, 17)\t1.0\n",
      "  (18, 18)\t1.0\n",
      "  (19, 19)\t1.0\n",
      "  (20, 20)\t1.0\n",
      "  (21, 21)\t1.0\n",
      "  (22, 22)\t1.0\n",
      "  (23, 23)\t1.0\n",
      "  (24, 24)\t1.0\n",
      "  :\t:\n",
      "  (2908, 3274)\t0.9999999999999998\n",
      "  (1869, 1515)\t0.9999999999999998\n",
      "  (1515, 1869)\t0.9999999999999998\n",
      "  (1051, 1280)\t0.9999999999999998\n",
      "  (1280, 1051)\t0.9999999999999998\n",
      "  (3083, 1610)\t0.9999999999999998\n",
      "  (1610, 3083)\t0.9999999999999998\n",
      "  (1235, 435)\t-0.04174281594577362\n",
      "  (435, 1235)\t-0.04174281594577362\n",
      "  (1092, 360)\t0.9999999999999998\n",
      "  (360, 1092)\t0.9999999999999998\n",
      "  (317, 1130)\t0.9999999999999998\n",
      "  (1130, 317)\t0.9999999999999998\n",
      "  (3081, 2697)\t0.9999999999999998\n",
      "  (2697, 3081)\t0.9999999999999998\n",
      "  (211, 987)\t-0.05383239994357515\n",
      "  (987, 211)\t-0.05383239994357515\n",
      "  (1678, 3386)\t0.0015251141537247846\n",
      "  (3386, 1678)\t0.0015251141537247846\n",
      "  (2110, 2911)\t-0.030100673174225594\n",
      "  (2911, 2110)\t-0.030100673174225594\n",
      "  (2732, 1291)\t-0.014285421539392362\n",
      "  (1291, 2732)\t-0.014285421539392362\n",
      "  (1825, 1450)\t0.9999999999999998\n",
      "  (1450, 1825)\t0.9999999999999998\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(13418, 13418)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(eventSimMatrix_eventBased)\n",
    "eventSimMatrix_eventBased.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 将所有特征串联起来，构成RS_Train.csv\n",
    "#RS_Test.csv\n",
    "#为最后推荐系统做准备\n",
    "\n",
    "\n",
    "class RecommonderSystem:\n",
    "  def __init__(self):\n",
    "    # 读入数据做初始化\n",
    "    \n",
    "    #用户和活动新的索引\n",
    "    self.userIndex = pickle.load(open(\"PE_userIndex.pkl\", 'rb'))\n",
    "    self.eventIndex = pickle.load(open(\"PE_eventIndex.pkl\", 'rb'))\n",
    "    self.n_users = len(self.userIndex)\n",
    "    self.n_items = len(self.eventIndex)\n",
    "    \n",
    "    #用户-活动关系矩阵R\n",
    "    self.userEventScores = sio.mmread(\"PE_userEventScores\").todense()\n",
    "    \n",
    "    #倒排表\n",
    "    ##每个用户参加的事件\n",
    "    self.itemsForUser = pickle.load(open(\"PE_eventsForUser.pkl\", 'rb'))\n",
    "    ##事件参加的用户\n",
    "    self.usersForItem = pickle.load(open(\"PE_usersForEvent.pkl\", 'rb'))\n",
    "    \n",
    "    #基于模型的协同过滤参数初始化,训练\n",
    "    self.init_SVD()\n",
    "    self.train_SVD(trainfile = \"train.csv\")\n",
    "    \n",
    "    #根据用户属性计算出的用户之间的相似度\n",
    "    self.userSimMatrix = sio.mmread(\"US_userSimMatrix\").todense()\n",
    "    \n",
    "    #根据活动属性计算出的活动之间的相似度\n",
    "    self.eventPropSim = sio.mmread(\"EV_eventPropSim\").todense()\n",
    "    self.eventContSim = sio.mmread(\"EV_eventContSim\").todense()\n",
    "    \n",
    "    #每个用户的朋友的数目\n",
    "    self.numFriends = sio.mmread(\"UF_numFriends\")\n",
    "    #用户的每个朋友参加活动的分数对该用户的影响\n",
    "    self.userFriends = sio.mmread(\"UF_userFriends\").todense()\n",
    "    \n",
    "    #活动本身的热度\n",
    "    self.eventPopularity = sio.mmread(\"EA_eventPopularity\").todense()\n",
    "    \n",
    "    # 先读取两个相似度的矩阵，存在变量缓存中，不用每次调用函数都读一次，这样很慢\n",
    "    self.userSimMatrix_users = sio.mmread(\"userSimMatrix_userBased\").todense()\n",
    "    self.userSimMatrix_events = sio.mmread(\"eventSimMatrix_eventBased\").todense()\n",
    "    \n",
    "  def init_SVD(self, K=20):\n",
    "    #初始化模型参数（for 基于模型的协同过滤SVD_CF）\n",
    "    self.K = K                       # K一般取值为20 -- 100，此处取20\n",
    "    \n",
    "    #init parameters\n",
    "    #bias\n",
    "    self.bi = np.zeros(self.n_items)  # 记录活动本身的特性，作为偏差\n",
    "    self.bu = np.zeros(self.n_users)  # 记录用户本身的特性，作为偏差\n",
    "    \n",
    "    #the small matrix\n",
    "    self.P = random((self.n_users,self.K))/10*(np.sqrt(self.K))\n",
    "    self.Q = random((self.K, self.n_items))/10*(np.sqrt(self.K))  \n",
    "                  \n",
    "          \n",
    "  def train_SVD(self,trainfile = 'train.csv', steps=100,gamma=0.04,Lambda=0.15):\n",
    "    #训练SVD模型（for 基于模型的协同过滤SVD_CF）\n",
    "    #gamma：为学习率\n",
    "    #Lambda：正则参数\n",
    "    \n",
    "    #偷懒了，为了和原来的代码的输入接口一样，直接从训练文件中去读取数据\n",
    "    print (\"SVD Train...\")\n",
    "    ftrain = open(trainfile, 'r')\n",
    "    ftrain.readline()\n",
    "    self.mu = 0.0\n",
    "    n_records = 0\n",
    "    uids = []  #每条记录的用户索引\n",
    "    i_ids = [] #每条记录的item索引\n",
    "    #用户-Item关系矩阵R（内容同userEventScores相同），临时变量，训练完了R不再需要\n",
    "    R = np.zeros((self.n_users, self.n_items))\n",
    "    \n",
    "    for line in ftrain:\n",
    "        cols = line.strip().split(\",\")\n",
    "        u = self.userIndex[cols[0]]  #用户\n",
    "        i = self.eventIndex[cols[1]] #活动\n",
    "        \n",
    "        uids.append(u)         # 得出训练集中的用户集合\n",
    "        i_ids.append(i)        # 得出训练集中的活动集合\n",
    "        \n",
    "        R[u,i] = int(cols[4])  #interested  分数\n",
    "        self.mu += R[u,i]\n",
    "        n_records += 1\n",
    "    \n",
    "    ftrain.close()\n",
    "    self.mu /= n_records       #计算所有打分的均值，公式中需使用\n",
    "    \n",
    "    # 我感觉真正在做模型训练时，学习率和正则参数应该都是超参数，需要调参的，并且这两个参数非常重要，学习率一般采用\n",
    "    # 随迭代次数增加逐渐减小的方法，既能在前期快速收敛，也能在训练后期防止overshot，但老师这边直接给出了，就这样用好了\n",
    "    # steps应该是终止条件，终止条件可以是迭代次数小于某个值，预测误差小于某个阈值，梯度下降幅度小于某个阈值\n",
    "    \n",
    "    \n",
    "    # 各个参数已经有了初始值\n",
    "    # 目标函数SSE = 预测残差的平方 + P的L2正则 + Q的L2正则 + bi的L2正则 + bu的L2正则\n",
    "\n",
    "    for n_round in range(steps):\n",
    "        for n_count in range(n_records):\n",
    "            pred_score = self.pred_SVD(uids[n_count],i_ids[n_count])\n",
    "            e_ui = R[uids[n_count], i_ids[n_count]] - pred_score\n",
    "            \n",
    "            # 求目标函数中各参数的偏导数\n",
    "            gSSE_gPuk = -e_ui * self.Q[:,i_ids[n_count]] + Lambda * self.P[uids[n_count],:] #对 Puk 求偏导，维数 K\n",
    "            gSSE_gQki = -e_ui * self.P[uids[n_count],:] + Lambda * self.Q[:,i_ids[n_count]] #对 Qki 求偏导，维数 K\n",
    "            gSSE_gBu = -e_ui + Lambda * self.bu[uids[n_count]]                              #对 Bu 求偏导，维数 1\n",
    "            gSSE_gBi = -e_ui + Lambda * self.bi[i_ids[n_count]]                             #对 Bi 求偏导，维数 1\n",
    "            \n",
    "            # 更新迭代参数,每个参数都是负梯度方向下降\n",
    "            self.P[uids[n_count],:] = self.P[uids[n_count],:] - gamma * gSSE_gPuk\n",
    "            self.Q[:,i_ids[n_count]] = self.Q[:,i_ids[n_count]] - gamma * gSSE_gQki\n",
    "            self.bu[uids[n_count]] = self.bu[uids[n_count]] - gamma * gSSE_gBu\n",
    "            self.bi[i_ids[n_count]] = self.bi[i_ids[n_count]] - gamma * gSSE_gBi \n",
    "            \n",
    "            # 打印部分训练信息，反应代码当前跑的进度，但不能打的太多，会很卡很乱\n",
    "            if n_round%10==0 and n_count%5000==0:\n",
    "                print(\"n_round = %d, n_count = %d\" % (n_round, n_count))\n",
    "                print(self.P[uids[n_count],:])\n",
    "                print(self.Q[:,i_ids[n_count]])\n",
    "                print(self.bu[uids[n_count]])\n",
    "                print(self.bi[i_ids[n_count]])\n",
    "    \n",
    "    print (\"SVD trained\") \n",
    "    \n",
    "    # 先确定目标函数，目的是求最优解，故需要求对各个参数的偏导数，通过沿负梯度下降的方法来使目标函数收敛到最优值（应该是局部最优值）\n",
    "    # 求偏导 + 更新迭代参数 ，这两部可以融合在一起，但分开更清晰\n",
    "\n",
    "  def pred_SVD(self, uid, i_id):\n",
    "    #根据当前参数，预测用户uid对Item（i_id）的打分  \n",
    "    # 评分 = 均值 + 活动属性偏差 + 用户属性偏差 + 用户和活动的关联属性\n",
    "    ans=self.mu + self.bi[i_id] + self.bu[uid] + np.dot(self.P[uid,:],self.Q[:,i_id])  \n",
    "        \n",
    "    #将打分范围控制在0-1之间\n",
    "    if ans>1:  \n",
    "        return 1  \n",
    "    elif ans<0:  \n",
    "        return 0\n",
    "    return ans\n",
    "\n",
    "  def sim_cal_UserCF(self, uid1, uid2 ):\n",
    "    # uid1 和 uid2 是两个user重编码的序号，不是真正的ID\n",
    "    similarity = 0.0\n",
    "    \n",
    "    # 计算两个用户之间相似度时直接查计算好的相似度矩阵，并返回相应值\n",
    "    #userSimMatrix_users = sio.mmread(\"userSimMatrix_userBased\").todense()\n",
    "    # 原本是直接通过读取相似度矩阵并查询，后来代码跑的时间很长，原因是这个函数调用的次数非常频繁，但如果每次都重新读取矩阵文件，再\n",
    "    # 将数据放入缓存，会造成很多时间消耗，而且缓存不释放，占用电脑内存，导致后面代码跑了一个多小时，只生成了81条数据就不动了\n",
    "    # 优化：在初始化的时候读取一次文件，并保存在缓存中，每次查询缓存里的数据即可\n",
    "    \n",
    "    similarity = self.userSimMatrix_users[uid1, uid2]\n",
    "    #print(\"sim_cal_UserCF similarity = %d\"%similarity)  \n",
    "    return similarity  \n",
    "\n",
    "  def userCFReco(self, userId, eventId):\n",
    "    \"\"\"\n",
    "    根据User-based协同过滤，得到event的推荐度\n",
    "    基本的伪代码思路如下：\n",
    "    for item i\n",
    "      for every other user v that has a preference for i\n",
    "        compute similarity s between u and v\n",
    "        incorporate v's preference for i weighted by s into running average\n",
    "    return top items ranked by weighted average\n",
    "    \"\"\"\n",
    "    \n",
    "    ans = 0.0\n",
    "\n",
    "    u = self.userIndex[userId]\n",
    "    i = self.eventIndex[eventId]\n",
    "    # 获取参加该活动的所有用户（相关联用户协助预测）\n",
    "    # 如果关联用户很多，可以考虑设置相似度的阈值或邻居人数，选取部分用户计算推荐度\n",
    "    relevant_users = self.usersForItem[i]\n",
    "    # 公式中的分子和分母（计算预测公式）\n",
    "    numerator_1 = 0.0\n",
    "    denominator_1 = 0.0\n",
    "    # 计算涉及到的 用户1和用户2的相似度、用户2相对于平均打分的偏置 在上面已计算完成，直接调用\n",
    "    for user0 in relevant_users:\n",
    "        numerator_1 = numerator_1 + self.sim_cal_UserCF(u, user0) * row_userEventScores[user0,i]\n",
    "        denominator_1 = denominator_1 + self.sim_cal_UserCF(u, user0)\n",
    "     \n",
    "    #增加对分母的判断，避免分母为0的情况，如果分母为0说明两者相似度系数为0\n",
    "    if denominator_1==0:\n",
    "        ans = 0\n",
    "    else:\n",
    "        ans = mean_score[u] + numerator_1 / denominator_1\n",
    "        \n",
    "    #print(\"UserCFReco ans = %d\"%ans)  \n",
    "    return ans\n",
    "\n",
    "\n",
    "  def sim_cal_ItemCF(self, i_id1, i_id2):\n",
    "        \n",
    "    #计算Item i_id1和i_id2之间的相似性\n",
    "    # i_id1 和 i_id2 是重编码的活动序号，不是原始ID\n",
    "\n",
    "    similarity = 0.0\n",
    "\n",
    "    #userSimMatrix_events = sio.mmread(\"eventSimMatrix_eventBased\").todense()\n",
    "    # 上面读文件的代码挖的坑同用户协同过滤一样，不能这样做\n",
    "    similarity = self.userSimMatrix_events[i_id1, i_id2]\n",
    "    #print(\"sim_cal_ItemCF similarity = %d\"%similarity)  \n",
    "    return similarity     \n",
    "            \n",
    "  def eventCFReco(self, userId, eventId):    \n",
    "    \"\"\"\n",
    "    根据基于物品的协同过滤，得到Event的推荐度\n",
    "    基本的伪代码思路如下：\n",
    "    for item i \n",
    "        for every item j tht u has a preference for\n",
    "            compute similarity s between i and j\n",
    "            add u's preference for j weighted by s to a running average\n",
    "    return top items, ranked by weighted average\n",
    "    \"\"\"\n",
    "    ans = 0.0\n",
    "\n",
    "    u = self.userIndex[userId]\n",
    "    i = self.eventIndex[eventId]\n",
    "    # 获取该用户参加的所有活动（相关联活动协助预测）\n",
    "    relevant_events = self.itemsForUser[u]\n",
    "    # 预测公式中的分子和分母\n",
    "    numerator_2 = 0.0\n",
    "    denominator_2 = 0.0\n",
    "    # 计算涉及到的 活动1和用活动2的相似度等参数 在上面已计算完成，直接调用\n",
    "    # 注意，这里的分子是活动的相似度 乘以 用户对该活动的打分，而不是（打分-均值），这个跟上面基于用户的协同过滤得到推荐度有些差别\n",
    "    # 老师在预测的时候，选取了相似度最高的两个item，我这边选择整行的活动一起计算，因为无关的活动相似度为0，关系不紧密的活动，相似度\n",
    "    # 加权值也会比较小，所以应该影响不大\n",
    "    for event0 in relevant_events:\n",
    "        numerator_2 = numerator_2 + self.sim_cal_ItemCF(i, event0) * self.userEventScores[u,event0]\n",
    "        denominator_2 = denominator_2 + self.sim_cal_ItemCF(i, event0)\n",
    "    \n",
    "    # 规避分母为0的情况\n",
    "    if denominator_2==0:\n",
    "        ans = 0\n",
    "    else:    \n",
    "        ans = numerator_2 / denominator_2\n",
    "        \n",
    "    #print(\"eventCFReco ans = %d\"%ans)  \n",
    "    return ans\n",
    "    \n",
    "  def svdCFReco(self, userId, eventId):\n",
    "    #基于模型的协同过滤, SVD++/LFM\n",
    "    u = self.userIndex[userId]\n",
    "    i = self.eventIndex[eventId]\n",
    "\n",
    "    return self.pred_SVD(u,i)\n",
    "\n",
    "  def userReco(self, userId, eventId):\n",
    "    \"\"\"\n",
    "    类似基于User-based协同过滤，只是用户之间的相似度由用户本身的属性得到，计算event的推荐度\n",
    "    基本的伪代码思路如下：\n",
    "    for item i\n",
    "      for every other user v that has a preference for i\n",
    "        compute similarity s between u and v\n",
    "        incorporate v's preference for i weighted by s into running aversge\n",
    "    return top items ranked by weighted average\n",
    "    \"\"\"\n",
    "    i = self.userIndex[userId]\n",
    "    j = self.eventIndex[eventId]\n",
    "\n",
    "    vs = self.userEventScores[:, j]\n",
    "    sims = self.userSimMatrix[i, :]\n",
    "\n",
    "    prod = sims * vs\n",
    "\n",
    "    try:\n",
    "      return prod[0, 0] - self.userEventScores[i, j]\n",
    "    except IndexError:\n",
    "      return 0\n",
    "\n",
    "  def eventReco(self, userId, eventId):\n",
    "    \"\"\"\n",
    "    类似基于Item-based协同过滤，只是item之间的相似度由item本身的属性得到，计算Event的推荐度\n",
    "    基本的伪代码思路如下：\n",
    "    for item i \n",
    "      for every item j that u has a preference for\n",
    "        compute similarity s between i and j\n",
    "        add u's preference for j weighted by s to a running average\n",
    "    return top items, ranked by weighted average\n",
    "    \"\"\"\n",
    "    i = self.userIndex[userId]\n",
    "    j = self.eventIndex[eventId]\n",
    "    js = self.userEventScores[i, :]\n",
    "    psim = self.eventPropSim[:, j]\n",
    "    csim = self.eventContSim[:, j]\n",
    "    pprod = js * psim\n",
    "    cprod = js * csim\n",
    "    \n",
    "    pscore = 0\n",
    "    cscore = 0\n",
    "    try:\n",
    "      pscore = pprod[0, 0] - self.userEventScores[i, j]\n",
    "    except IndexError:\n",
    "      pass\n",
    "    try:\n",
    "      cscore = cprod[0, 0] - self.userEventScores[i, j]\n",
    "    except IndexError:\n",
    "      pass\n",
    "    return pscore, cscore\n",
    "\n",
    "  def userPop(self, userId):\n",
    "    \"\"\"\n",
    "    基于用户的朋友个数来推断用户的社交程度\n",
    "    主要的考量是如果用户的朋友非常多，可能会更倾向于参加各种社交活动\n",
    "    \"\"\"\n",
    "    if userId in self.userIndex:\n",
    "      i = self.userIndex[userId]\n",
    "      try:\n",
    "        return self.numFriends[0, i]\n",
    "      except IndexError:\n",
    "        return 0\n",
    "    else:\n",
    "      return 0\n",
    "\n",
    "  def friendInfluence(self, userId):\n",
    "    \"\"\"\n",
    "    朋友对用户的影响\n",
    "    主要考虑用户所有的朋友中，有多少是非常喜欢参加各种社交活动/event的\n",
    "    用户的朋友圈如果都积极参与各种event，可能会对当前用户有一定的影响\n",
    "    \"\"\"\n",
    "    nusers = np.shape(self.userFriends)[1]\n",
    "    i = self.userIndex[userId]\n",
    "    return (self.userFriends[i, :].sum(axis=0) / nusers)[0,0]\n",
    "\n",
    "  def eventPop(self, eventId):\n",
    "    \"\"\"\n",
    "    本活动本身的热度\n",
    "    主要是通过参与的人数来界定的\n",
    "    \"\"\"\n",
    "    i = self.eventIndex[eventId]\n",
    "    return self.eventPopularity[i, 0]\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generateRSData(RS, train=True, header=True):\n",
    "    \"\"\"\n",
    "    把前面user-based协同过滤 和 item-based协同过滤，以及各种热度和影响度作为特征组合在一起\n",
    "    生成新的训练数据，用于分类器分类使用\n",
    "    \"\"\"\n",
    "    fn = \"train.csv\" if train else \"test.csv\"\n",
    "    fin = open(fn, 'r')\n",
    "    fout = open(\"RS_\" + fn, 'w')\n",
    "    \n",
    "    #忽略第一行（列名字）\n",
    "    fin.readline().strip().split(\",\")\n",
    "    \n",
    "    # write output header\n",
    "    if header:\n",
    "      ocolnames = [\"invited\", \"userCF_reco\", \"evtCF_reco\",\"svdCF_reco\",\"user_reco\", \"evt_p_reco\",\n",
    "        \"evt_c_reco\", \"user_pop\", \"frnd_infl\", \"evt_pop\"]\n",
    "      if train:\n",
    "        ocolnames.append(\"interested\")\n",
    "        ocolnames.append(\"not_interested\")\n",
    "      fout.write(\",\".join(ocolnames) + \"\\n\")\n",
    "    \n",
    "    ln = 0\n",
    "    for line in fin:\n",
    "      ln += 1\n",
    "      if ln%500 == 0:\n",
    "          print (\"%s:%d (userId, eventId)=(%s, %s)\" % (fn, ln, userId, eventId))\n",
    "          #break;\n",
    "      \n",
    "      cols = line.strip().split(\",\")\n",
    "      userId = cols[0]\n",
    "      eventId = cols[1]\n",
    "      invited = cols[2]\n",
    "      \n",
    "      userCF_reco = RS.userCFReco(userId, eventId)\n",
    "      itemCF_reco = RS.eventCFReco(userId, eventId)\n",
    "      svdCF_reco = RS.svdCFReco(userId, eventId)\n",
    "        \n",
    "      user_reco = RS.userReco(userId, eventId)\n",
    "      evt_p_reco, evt_c_reco = RS.eventReco(userId, eventId)\n",
    "      user_pop = RS.userPop(userId)\n",
    "     \n",
    "      frnd_infl = RS.friendInfluence(userId)\n",
    "      evt_pop = RS.eventPop(eventId)\n",
    "      ocols = [invited, userCF_reco, itemCF_reco, svdCF_reco,user_reco, evt_p_reco,\n",
    "        evt_c_reco, user_pop, frnd_infl, evt_pop]\n",
    "      \n",
    "      if train:\n",
    "        ocols.append(cols[4]) # interested\n",
    "        ocols.append(cols[5]) # not_interested\n",
    "      fout.write(\",\".join(map(lambda x: str(x), ocols)) + \"\\n\")\n",
    "    \n",
    "    fin.close()\n",
    "    fout.close()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SVD Train...\n",
      "n_round = 0, n_count = 0\n",
      "[0.08387341 0.00764184 0.18967457 0.36717707 0.32978268 0.26320094\n",
      " 0.2524052  0.32860424 0.36969414 0.40404306 0.33787495 0.25839888\n",
      " 0.16233282 0.240746   0.15007781 0.18397874 0.3429503  0.14587543\n",
      " 0.27961638 0.0084594 ]\n",
      "[0.33902048 0.41535043 0.20392093 0.37908884 0.03614334 0.21029504\n",
      " 0.2158121  0.35005179 0.0177824  0.12527124 0.34852902 0.41850682\n",
      " 0.27026183 0.06534529 0.35433595 0.03040684 0.35947218 0.35630205\n",
      " 0.23963924 0.02939353]\n",
      "-0.04\n",
      "-0.04\n",
      "n_round = 0, n_count = 5000\n",
      "[ 0.40884093  0.14445671  0.16743264  0.06706455  0.31215671 -0.00091337\n",
      "  0.19685357  0.23720289  0.02571676  0.21753718  0.08901758  0.26400879\n",
      "  0.21370744  0.20000785  0.14633433  0.12323477  0.29431092  0.34773299\n",
      "  0.38058358  0.16211579]\n",
      "[0.02032378 0.23795501 0.13510514 0.38714766 0.33492045 0.05128374\n",
      " 0.1559093  0.40607224 0.37415957 0.03090713 0.1711644  0.26187507\n",
      " 0.1940971  0.27872118 0.03041138 0.27388829 0.16960878 0.06738418\n",
      " 0.25340429 0.20102576]\n",
      "-0.07952144\n",
      "-0.04\n",
      "n_round = 0, n_count = 10000\n",
      "[ 0.23177448  0.18476688  0.13392424  0.18844056 -0.01297552 -0.01009785\n",
      "  0.29428622  0.25792085  0.19993951 -0.09643861  0.20046993  0.20691451\n",
      "  0.01273443  0.11045433 -0.02502639  0.2862818   0.09043632 -0.06191902\n",
      " -0.08574507  0.06878986]\n",
      "[0.40440874 0.3747638  0.21379961 0.313741   0.22559732 0.31802291\n",
      " 0.36962944 0.14454382 0.20738603 0.02719723 0.26987886 0.12259436\n",
      " 0.32256543 0.1880432  0.228305   0.27807258 0.258236   0.10601989\n",
      " 0.14334379 0.14118171]\n",
      "-0.4368906499047448\n",
      "0.02442329813494501\n",
      "n_round = 0, n_count = 15000\n",
      "[0.10031015 0.41681658 0.11310041 0.39600885 0.17704957 0.39827889\n",
      " 0.3951408  0.34346869 0.11761434 0.0654508  0.37290156 0.07088011\n",
      " 0.29521487 0.42658202 0.17736548 0.29717603 0.4147897  0.17071098\n",
      " 0.20072159 0.05426311]\n",
      "[0.05416523 0.07825035 0.04769619 0.2352624  0.33455877 0.406145\n",
      " 0.39941669 0.07189101 0.27605071 0.11580539 0.03252727 0.00978427\n",
      " 0.14929251 0.14046374 0.03000152 0.017804   0.38857357 0.16676918\n",
      " 0.09887979 0.34205476]\n",
      "-0.04\n",
      "-0.04\n",
      "n_round = 10, n_count = 0\n",
      "[-0.00532926 -0.09651824  0.06533726  0.17817394  0.16765997  0.09707685\n",
      "  0.1246898   0.16367225  0.17505828  0.24247954  0.11844913  0.076142\n",
      "  0.05448908  0.08128332 -0.00087684  0.09368301  0.16923586 -0.01311344\n",
      "  0.10634252 -0.03408779]\n",
      "[ 0.30484174  0.35342071  0.14557148  0.28145956 -0.00176778  0.15201521\n",
      "  0.15002819  0.29225194 -0.04043935  0.05777495  0.29393363  0.3237859\n",
      "  0.19728492  0.04423639  0.28167586 -0.0085877   0.27113441  0.29684776\n",
      "  0.18559617  0.01677877]\n",
      "-0.24920147961091127\n",
      "-0.2323105568564293\n",
      "n_round = 10, n_count = 5000\n",
      "[ 0.16625282  0.04123664  0.07550274 -0.07494429  0.10659785  0.00901085\n",
      "  0.06179469  0.09985677  0.01075769  0.12440157  0.03816304  0.02383151\n",
      "  0.07084879  0.00406992  0.06785714  0.0255466   0.11389385  0.2071542\n",
      "  0.20822823  0.06107393]\n",
      "[-0.01280267  0.20184205  0.10628232  0.34635386  0.27491965  0.04828822\n",
      "  0.12650939  0.34326235  0.33236757  0.00971614  0.14686947  0.21717107\n",
      "  0.15873046  0.23851855  0.01802038  0.2370125   0.12774499  0.02976393\n",
      "  0.1914008   0.16645954]\n",
      "-0.2145713475093935\n",
      "-0.14579538918031618\n",
      "n_round = 10, n_count = 10000\n",
      "[ 0.03815894  0.01053554 -0.03125982  0.0248601   0.03586299  0.00730775\n",
      " -0.00030076 -0.01527528  0.02724504 -0.0447109  -0.00986451 -0.07625494\n",
      "  0.0292531  -0.01016165 -0.01307424  0.01222608 -0.02683763 -0.05068897\n",
      " -0.04218666  0.02593707]\n",
      "[0.39452113 0.35816203 0.19673422 0.30586042 0.21978058 0.29914162\n",
      " 0.35453469 0.13929404 0.20668337 0.01039741 0.2563478  0.10297988\n",
      " 0.30806674 0.17666546 0.21010259 0.27246529 0.23716093 0.08480942\n",
      " 0.12084944 0.14073711]\n",
      "-0.04179806115248791\n",
      "0.27798545144324355\n",
      "n_round = 10, n_count = 15000\n",
      "[ 0.09749134  0.28688585  0.06804385  0.24409905  0.10505704  0.2059247\n",
      "  0.21656292  0.19742327 -0.01449378  0.0524019   0.27638897  0.0743577\n",
      "  0.13117614  0.21158146  0.08468642  0.21522521  0.26383317  0.07563221\n",
      "  0.12619333 -0.07559907]\n",
      "[ 0.03855068  0.02055039  0.03386438  0.17374852  0.29348974  0.33677255\n",
      "  0.33089048  0.02762162  0.25461487  0.10051335 -0.0187462  -0.00087379\n",
      "  0.10993717  0.08401829  0.00978506 -0.02192164  0.31357052  0.14122368\n",
      "  0.06973797  0.32550136]\n",
      "-0.25023508386273785\n",
      "-0.191890893956212\n",
      "n_round = 20, n_count = 0\n",
      "[-0.01347392 -0.09488527  0.04192816  0.11070468  0.10081066  0.05062927\n",
      "  0.0833555   0.10857845  0.11570653  0.17046918  0.0623337   0.04168127\n",
      "  0.04179066  0.03272865 -0.01504845  0.06037913  0.10893345 -0.02097248\n",
      "  0.05835898 -0.02092451]\n",
      "[ 0.27151375  0.31369395  0.12722592  0.24715041 -0.00244376  0.13357732\n",
      "  0.13030566  0.25972487 -0.0385898   0.04874311  0.26119478  0.28461568\n",
      "  0.17226857  0.03969598  0.24858312 -0.00941753  0.23852338  0.26205328\n",
      "  0.16376828  0.01512635]\n",
      "-0.14165658542506915\n",
      "-0.22552989491718603\n",
      "n_round = 20, n_count = 5000\n",
      "[ 0.09995573  0.02441706  0.0480638  -0.0692138   0.0587708   0.02967277\n",
      "  0.0448373   0.06984448  0.02641289  0.0963116   0.0363837  -0.01890082\n",
      "  0.0504354  -0.01405686  0.05968736  0.01920535  0.06173851  0.15237925\n",
      "  0.1391764   0.03601426]\n",
      "[-0.01774894  0.17735945  0.0912129   0.3107267   0.23977089  0.04177464\n",
      "  0.10957489  0.30017833  0.29364723  0.00315982  0.12833953  0.19264389\n",
      "  0.13777045  0.21185004  0.01282726  0.20904055  0.10905369  0.01754303\n",
      "  0.16118503  0.14522179]\n",
      "-0.11040789574128286\n",
      "-0.1791255502098238\n",
      "n_round = 20, n_count = 10000\n",
      "[ 0.029023    0.00948876 -0.02561013  0.01518441  0.02234193  0.00278211\n",
      " -0.00113198 -0.01255503  0.02111644 -0.0321209  -0.00988334 -0.06347175\n",
      "  0.02158848 -0.01046585 -0.01545115  0.00766321 -0.01881119 -0.0409786\n",
      " -0.03411366  0.01843787]\n",
      "[0.37597453 0.33791046 0.17980617 0.29041655 0.21115322 0.28159128\n",
      " 0.33262201 0.12835128 0.19808446 0.00329763 0.23889166 0.08467574\n",
      " 0.29351238 0.16405243 0.19472493 0.25732231 0.2187328  0.07182925\n",
      " 0.10697456 0.1358374 ]\n",
      "-0.03214583286429817\n",
      "0.43211922790678875\n",
      "n_round = 20, n_count = 15000\n",
      "[ 0.1179844   0.23349436  0.07430914  0.19780348  0.08212817  0.13687231\n",
      "  0.16796565  0.15217834 -0.0262312   0.04905307  0.23133206  0.07914351\n",
      "  0.0921333   0.13450342  0.06190794  0.17356009  0.19597824  0.07553496\n",
      "  0.11970136 -0.08571737]\n",
      "[ 0.03004268  0.00478104  0.0277701   0.15118829  0.27102452  0.307539\n",
      "  0.30073759  0.01626875  0.24087044  0.09171748 -0.03187504 -0.00523433\n",
      "  0.09732459  0.06960858  0.00514373 -0.03152396  0.28235163  0.12864055\n",
      "  0.05871369  0.31105639]\n",
      "-0.1262676645693177\n",
      "-0.236779699813802\n",
      "n_round = 30, n_count = 0\n",
      "[-0.01685823 -0.09195393  0.03004854  0.06432733  0.05746609  0.02129335\n",
      "  0.05461514  0.07076324  0.08197025  0.12214241  0.02969632  0.02266066\n",
      "  0.03585101  0.00623073 -0.01835025  0.03706657  0.06914376 -0.01947426\n",
      "  0.02905749 -0.00953829]\n",
      "[ 0.24163764  0.27902904  0.11192314  0.21819946 -0.00214561  0.11807185\n",
      "  0.11394569  0.23108141 -0.03544563  0.04211306  0.23229818  0.2511448\n",
      "  0.15119323  0.03583437  0.2202018  -0.00910388  0.21085788  0.23173929\n",
      "  0.14509942  0.01396296]\n",
      "-0.079764371623797\n",
      "-0.21614489162793943\n",
      "n_round = 30, n_count = 5000\n",
      "[ 0.06581163  0.01603499  0.03257706 -0.05712881  0.0356385   0.03400361\n",
      "  0.03443736  0.05116274  0.02869306  0.07519932  0.03161862 -0.02983741\n",
      "  0.03812561 -0.01671161  0.05068942  0.01526574  0.03660083  0.1149048\n",
      "  0.09797922  0.02314025]\n",
      "[-0.01908049  0.15642384  0.07921984  0.27811551  0.21067841  0.03570696\n",
      "  0.09553697  0.26366285  0.25917527 -0.00072439  0.11238026  0.17187497\n",
      "  0.12033959  0.18850411  0.009111    0.18463286  0.09470939  0.01007178\n",
      "  0.13805082  0.12755124]\n",
      "-0.062019437310179316\n",
      "-0.20038361718258338\n",
      "n_round = 30, n_count = 10000\n",
      "[ 0.02242641  0.0074547  -0.02093907  0.00882081  0.01376143  0.00046717\n",
      " -0.00160095 -0.0103021   0.01635264 -0.02360939 -0.00936776 -0.05082089\n",
      "  0.01450129 -0.0108089  -0.01622236  0.00670003 -0.01252164 -0.03389457\n",
      " -0.02856951  0.01343104]\n",
      "[ 3.56476298e-01  3.18695780e-01  1.66342154e-01  2.74415445e-01\n",
      "  2.00580123e-01  2.64918224e-01  3.12569239e-01  1.19347327e-01\n",
      "  1.88415028e-01 -1.17031179e-04  2.23475273e-01  7.29537844e-02\n",
      "  2.78042959e-01  1.52988585e-01  1.81203344e-01  2.42759987e-01\n",
      "  2.03852505e-01  6.31756712e-02  9.69275866e-02  1.29548904e-01]\n",
      "-0.025247261556570404\n",
      "0.5239235448031038\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "n_round = 30, n_count = 15000\n",
      "[ 0.11987733  0.1897295   0.06992008  0.16088493  0.06313843  0.08775223\n",
      "  0.13037113  0.1185874  -0.03243466  0.04151125  0.19364084  0.07483658\n",
      "  0.066461    0.08497     0.04487639  0.13802234  0.14344927  0.07173877\n",
      "  0.11299174 -0.08294418]\n",
      "[ 0.02314919 -0.00445632  0.02304413  0.13474135  0.2520791   0.28484883\n",
      "  0.27682671  0.00960613  0.22803877  0.08441104 -0.03901948 -0.00823336\n",
      "  0.08830372  0.06098186  0.00260084 -0.03626562  0.25867787  0.11795012\n",
      "  0.0503015   0.29646257]\n",
      "-0.036835150770831916\n",
      "-0.2655325726376718\n",
      "n_round = 40, n_count = 0\n",
      "[-0.01939991 -0.09010556  0.02356377  0.03170321  0.02782507  0.00096918\n",
      "  0.03438786  0.04420995  0.0606652   0.08866271  0.00835139  0.01068668\n",
      "  0.03296034 -0.00999031 -0.01851693  0.01989972  0.04212044 -0.01633555\n",
      "  0.00940904 -0.00157668]\n",
      "[ 0.21518744  0.24871215  0.09869291  0.1931606  -0.00135041  0.10468494\n",
      "  0.09978071  0.20601851 -0.03209788  0.03678675  0.2069401   0.22190124\n",
      "  0.13275399  0.03259822  0.19548078 -0.00839545  0.18688018  0.20506967\n",
      "  0.12894297  0.01316129]\n",
      "-0.05035676770247433\n",
      "-0.2092339858871447\n",
      "n_round = 40, n_count = 5000\n",
      "[ 0.04695689  0.01188496  0.02341763 -0.04569395  0.02419467  0.03296926\n",
      "  0.0277348   0.03948305  0.0271844   0.06011957  0.02730309 -0.02995997\n",
      "  0.03047229 -0.01480784  0.04318543  0.01303647  0.02381365  0.08941419\n",
      "  0.07265115  0.01663731]\n",
      "[-0.01887118  0.13819622  0.06925416  0.24839428  0.1857494   0.03047444\n",
      "  0.08361907  0.23217479  0.22878475 -0.00302072  0.09859778  0.15346481\n",
      "  0.1054925   0.16770008  0.00643319  0.16320031  0.08292438  0.00534436\n",
      "  0.11940185  0.11239104]\n",
      "-0.0392107946136211\n",
      "-0.21319347965791993\n",
      "n_round = 40, n_count = 10000\n",
      "[ 0.01782036  0.00556292 -0.01750774  0.00479307  0.00860149 -0.00065961\n",
      " -0.00191871 -0.00885593  0.01280666 -0.01823047 -0.00900628 -0.04135075\n",
      "  0.0093297  -0.01096187 -0.01620054  0.00646838 -0.00839807 -0.02894078\n",
      " -0.02469131  0.01042123]\n",
      "[ 0.33709203  0.30039713  0.15487528  0.25874414  0.18964304  0.24923395\n",
      "  0.29392973  0.11147712  0.1785034  -0.00186843  0.209474    0.06474095\n",
      "  0.26259253  0.14300612  0.16909442  0.22896521  0.1909245   0.05677398\n",
      "  0.08893389  0.12288976]\n",
      "-0.020305569907371528\n",
      "0.5784845345368566\n",
      "n_round = 40, n_count = 15000\n",
      "[ 0.11563147  0.1551421   0.0633209   0.13222701  0.04775078  0.05253229\n",
      "  0.10219545  0.09327205 -0.03534385  0.0336151   0.16334131  0.06831723\n",
      "  0.04904842  0.0511192   0.03225576  0.10960973  0.10398501  0.06748263\n",
      "  0.10670616 -0.07634179]\n",
      "[ 0.01747346 -0.01043566  0.01926213  0.12154945  0.23532517  0.26569577\n",
      "  0.2564298   0.00522279  0.21594522  0.07810701 -0.04320927 -0.01036754\n",
      "  0.08106596  0.05501021  0.00106722 -0.03861536  0.23909861  0.10850089\n",
      "  0.04334134  0.28202303]\n",
      "0.02350827109855523\n",
      "-0.28647371698617746\n",
      "n_round = 50, n_count = 0\n",
      "[-0.02180557 -0.08929812  0.0203677   0.00876916  0.00657691 -0.01369173\n",
      "  0.02014495  0.02553438  0.04661729  0.06492645 -0.00654199  0.00300325\n",
      "  0.03195438 -0.0208081  -0.01744772  0.00662696  0.02383791 -0.0131059\n",
      " -0.00428172  0.00352837]\n",
      "[ 1.91821224e-01  2.22093229e-01  8.72267455e-02  1.71492162e-01\n",
      " -1.33757883e-04  9.30851735e-02  8.75022080e-02  1.84215141e-01\n",
      " -2.86397616e-02  3.25967375e-02  1.84752841e-01  1.96296817e-01\n",
      "  1.16568074e-01  2.98895319e-02  1.73900296e-01 -7.40616356e-03\n",
      "  1.66102448e-01  1.81579081e-01  1.14980553e-01  1.26562280e-02]\n",
      "-0.037117387762852366\n",
      "-0.203787606800666\n",
      "n_round = 50, n_count = 5000\n",
      "[ 0.03590853  0.00977537  0.01784406 -0.03635426  0.01833057  0.03023389\n",
      "  0.02325497  0.03195048  0.02478899  0.04932271  0.02382006 -0.02670613\n",
      "  0.02549878 -0.01189868  0.03716048  0.01171288  0.01701116  0.07179708\n",
      "  0.0565065   0.01321711]\n",
      "[-0.01799451  0.1221901   0.06076479  0.22149426  0.16403437  0.02604353\n",
      "  0.07335588  0.20474672  0.20202997 -0.00436488  0.08662735  0.13694778\n",
      "  0.09266921  0.14910285  0.00446944  0.14431252  0.07289845  0.00225697\n",
      "  0.10387136  0.09918525]\n",
      "-0.02867710644142748\n",
      "-0.2201532262274851\n",
      "n_round = 50, n_count = 10000\n",
      "[ 0.01472304  0.00413457 -0.01505362  0.00236953  0.00541748 -0.00123982\n",
      " -0.00197005 -0.0079356   0.0102122  -0.01499962 -0.00878155 -0.03480375\n",
      "  0.00576171 -0.01090791 -0.01580415  0.00621386 -0.00576334 -0.02535073\n",
      " -0.02187124  0.00856588]\n",
      "[ 0.31832216  0.28303561  0.14469044  0.24374305  0.17893297  0.23450082\n",
      "  0.27650247  0.10436347  0.16875252 -0.00284525  0.1965573   0.05843102\n",
      "  0.24762509  0.13386316  0.15808041  0.2159126   0.179232    0.05164448\n",
      "  0.08216512  0.11627967]\n",
      "-0.017006628197729953\n",
      "0.6107507947913653\n",
      "n_round = 50, n_count = 15000\n",
      "[ 0.10952193  0.12838377  0.05708937  0.11026363  0.03564698  0.02773287\n",
      "  0.08174719  0.07424036 -0.03598624  0.02672197  0.13955053  0.06166732\n",
      "  0.03743346  0.02771116  0.02321176  0.08758357  0.07488087  0.06383403\n",
      "  0.10123615 -0.06875093]\n",
      "[ 1.27764005e-02 -1.44016805e-02  1.61777786e-02  1.10521264e-01\n",
      "  2.20218673e-01  2.48898643e-01  2.38467085e-01  2.22151076e-03\n",
      "  2.04481968e-01  7.25640825e-02 -4.55817282e-02 -1.18754783e-02\n",
      "  7.49391095e-02  5.05611484e-02  1.19984534e-04 -3.95312261e-02\n",
      "  2.22252095e-01  1.00012604e-01  3.74158411e-02  2.67883065e-01]\n",
      "0.06292004716090972\n",
      "-0.30225098516914\n",
      "n_round = 60, n_count = 0\n",
      "[-0.02427551 -0.08932274  0.01942939 -0.00754529 -0.009298   -0.02457514\n",
      "  0.00992051  0.01235411  0.03743084  0.04771698 -0.01766595 -0.0018539\n",
      "  0.03213913 -0.02879966 -0.01577116 -0.00405855  0.01152942 -0.01016893\n",
      " -0.01413309  0.00702162]\n",
      "[ 0.17121097  0.19872987  0.07725235  0.15267895  0.0013599   0.08298815\n",
      "  0.0768334   0.16523037 -0.02520757  0.02926734  0.16533079  0.17384867\n",
      "  0.10233221  0.02760399  0.15505863 -0.00624733  0.14803114  0.16090367\n",
      "  0.10288917  0.01238343]\n",
      "-0.03137835315166631\n",
      "-0.1997876961782637\n",
      "n_round = 60, n_count = 5000\n",
      "[ 0.02905932  0.00863218  0.01434885 -0.02911302  0.01515594  0.02719895\n",
      "  0.02015222  0.02688999  0.02242755  0.04148494  0.02106059 -0.02282774\n",
      "  0.02211048 -0.00914706  0.03236065  0.01083982  0.01320959  0.05936062\n",
      "  0.04581409  0.01126964]\n",
      "[-1.68479520e-02  1.08076719e-01  5.34264041e-02  1.97286515e-01\n",
      "  1.44968961e-01  2.22940114e-02  6.44381164e-02  1.80712059e-01\n",
      "  1.78458628e-01 -5.13024663e-03  7.61799886e-02  1.22111169e-01\n",
      "  8.15007036e-02  1.32491040e-01  3.00106085e-03  1.27634444e-01\n",
      "  6.42151399e-02  1.82539153e-04  9.06707342e-02  8.75961024e-02]\n",
      "-0.024262466876250607\n",
      "-0.22320816158073364\n",
      "n_round = 60, n_count = 10000\n",
      "[ 0.01256365  0.00314493 -0.01334078  0.00090536  0.00334902 -0.00158121\n",
      " -0.00188411 -0.00729136  0.0083658  -0.01309691 -0.00863587 -0.03032201\n",
      "  0.00327104 -0.01069467 -0.01530598  0.00582845 -0.00414185 -0.02269986\n",
      " -0.01974235  0.00733439]\n",
      "[ 0.30037612  0.26661547  0.13542235  0.22951692  0.16866077  0.22065493\n",
      "  0.26016364  0.09782046  0.15934911 -0.00344145  0.18453981  0.05322854\n",
      "  0.23332633  0.12541058  0.14794585  0.2035677   0.16844494  0.04730139\n",
      "  0.07620507  0.1098854 ]\n",
      "-0.014938799864182534\n",
      "0.6297327193483291\n",
      "n_round = 60, n_count = 15000\n",
      "[ 0.10305687  0.10780235  0.05172947  0.09332695  0.02629203  0.01059304\n",
      "  0.06714739  0.05982766 -0.03528946  0.02113987  0.12107156  0.05548039\n",
      "  0.02982499  0.01148521  0.01688209  0.07075179  0.05351261  0.06092485\n",
      "  0.09666874 -0.06122179]\n",
      "[ 8.88443842e-03 -1.70210827e-02  1.36267583e-02  1.01070416e-01\n",
      "  2.06437633e-01  2.33811133e-01  2.22342140e-01  1.32429970e-04\n",
      "  1.93579865e-01  6.76205606e-02 -4.67444219e-02 -1.29087087e-02\n",
      "  6.95765763e-02  4.70595020e-02 -4.67285019e-04 -3.95329821e-02\n",
      "  2.07393212e-01  9.23174335e-02  3.22998253e-02  2.54130492e-01]\n",
      "0.08817572359208424\n",
      "-0.31409237881483365\n",
      "n_round = 70, n_count = 0\n",
      "[-0.02686312 -0.09001954  0.01996755 -0.01923226 -0.02158199 -0.03291306\n",
      "  0.00250782  0.00296361  0.03143118  0.0350146  -0.02653684 -0.00491668\n",
      "  0.03306141 -0.03525126 -0.0139195  -0.01291677  0.00331829 -0.00769298\n",
      " -0.0215029   0.00948422]\n",
      "[ 0.15304723  0.17821654  0.06854045  0.13629771  0.0030192   0.07416965\n",
      "  0.06755683  0.14867495 -0.02189286  0.02659389  0.1483174   0.15414849\n",
      "  0.08980096  0.02566067  0.13859625 -0.00500028  0.13225638  0.14270782\n",
      "  0.09238996  0.01227305]\n",
      "-0.029524873097824526\n",
      "-0.19695456785188187\n",
      "n_round = 70, n_count = 5000\n",
      "[ 0.02458324  0.00795189  0.01207687 -0.02360354  0.01330868  0.02440064\n",
      "  0.01793263  0.0233518   0.02036881  0.03570416  0.01887604 -0.01927402\n",
      "  0.0197022  -0.006858    0.02855526  0.01020211  0.01096365  0.0503906\n",
      "  0.03846461  0.01004697]\n",
      "[-0.01562047  0.09560658  0.04702785  0.17559048  0.12816408  0.01910589\n",
      "  0.05664479  0.15957493  0.15766877 -0.00553825  0.06702855  0.10880655\n",
      "  0.07172285  0.11767444  0.00188255  0.11289073  0.05662427 -0.0012443\n",
      "  0.07930597  0.07738753]\n",
      "-0.023001485949569366\n",
      "-0.22376596166551022\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "n_round = 70, n_count = 10000\n",
      "[ 1.09739046e-02  2.50315819e-03 -1.21096433e-02  4.07702434e-05\n",
      "  1.96473890e-03 -1.79963122e-03 -1.69443531e-03 -6.81622048e-03\n",
      "  7.00447244e-03 -1.19966547e-02 -8.51307324e-03 -2.72056919e-02\n",
      "  1.52682152e-03 -1.03663353e-02 -1.47951623e-02  5.35779000e-03\n",
      " -3.17238333e-03 -2.07139472e-02 -1.80742338e-02  6.46636126e-03]\n",
      "[ 0.2833249   0.25111537  0.1268725   0.21607766  0.15889369  0.20763403\n",
      "  0.24482216  0.09174626  0.15037129 -0.00383946  0.17330599  0.04873247\n",
      "  0.21975395  0.11754968  0.13854655  0.19189976  0.15839582  0.04348844\n",
      "  0.07083059  0.10377147]\n",
      "-0.013815225842208031\n",
      "0.6408588590266506\n",
      "n_round = 70, n_count = 15000\n",
      "[ 0.09676808  0.09191823  0.0471461   0.08003374  0.01909983 -0.00107381\n",
      "  0.05676405  0.0487354  -0.03398455  0.0167753   0.10678084  0.04988001\n",
      "  0.02490725  0.0002001   0.01250734  0.05795576  0.03773648  0.05861442\n",
      "  0.09295289 -0.05418984]\n",
      "[ 0.00565206 -0.01871601  0.01149066  0.09282297  0.19376071  0.22003394\n",
      "  0.20767351 -0.00133314  0.18320199  0.06315798 -0.04708726 -0.0135802\n",
      "  0.06477363  0.04417233 -0.00083085 -0.03894946  0.19405343  0.08529472\n",
      "  0.02783485  0.24083042]\n",
      "0.10415521122631578\n",
      "-0.3228268781962252\n",
      "n_round = 80, n_count = 0\n",
      "[-0.02950954 -0.09118676  0.02144624 -0.02760996 -0.0313427  -0.03946181\n",
      " -0.00287511 -0.00381038  0.02753933  0.02551245 -0.03395872 -0.00681101\n",
      "  0.03443526 -0.0407603  -0.01211556 -0.02038147 -0.00205662 -0.00565313\n",
      " -0.02720409  0.01125909]\n",
      "[ 0.1370471   0.16019258  0.06089599  0.12199367  0.00475954  0.06644872\n",
      "  0.05948734  0.13421303 -0.01875866  0.0244215   0.13340392  0.1368415\n",
      "  0.0787622   0.02399799  0.12419352 -0.00372651  0.11843507  0.12668374\n",
      "  0.08324424  0.01226589]\n",
      "-0.029659912299829734\n",
      "-0.19497182107246275\n",
      "n_round = 80, n_count = 5000\n",
      "[ 0.0215143   0.00750456  0.01053779 -0.01941419  0.01214528  0.02200647\n",
      "  0.01630053  0.02078967  0.01865599  0.03136765  0.01714182 -0.01628688\n",
      "  0.01793024 -0.00503373  0.02554699  0.00970533  0.00955568  0.04377875\n",
      "  0.03323109  0.0092059 ]\n",
      "[-0.01440411  0.08457618  0.04141982  0.15619956  0.11332123  0.01637913\n",
      "  0.04980847  0.14094429  0.13931372 -0.0057225   0.05899103  0.09690024\n",
      "  0.06313402  0.10447605  0.00101645  0.09984769  0.04995494 -0.00224287\n",
      "  0.06944231  0.06837694]\n",
      "-0.023392393329395026\n",
      "-0.22280724789930215\n",
      "n_round = 80, n_count = 10000\n",
      "[ 0.00975724  0.00208989 -0.01121727 -0.00047086  0.00097457 -0.00196156\n",
      " -0.00147204 -0.00644725  0.0059842  -0.01138078 -0.00840342 -0.02497041\n",
      "  0.00025941 -0.00999153 -0.01432701  0.00483916 -0.0026085  -0.01917015\n",
      " -0.01674001  0.00580385]\n",
      "[ 0.26717653  0.2365      0.11892514  0.20340395  0.14964432  0.19538306\n",
      "  0.23040608  0.08607783  0.14184332 -0.00412852  0.16277661  0.04473079\n",
      "  0.20691176  0.11021203  0.12978522  0.18087884  0.14898839  0.04006298\n",
      "  0.06591431  0.09795972]\n",
      "-0.013313650010415554\n",
      "0.6473791166198593\n",
      "n_round = 80, n_count = 15000\n",
      "[ 0.09083703  0.07955993  0.04314211  0.06936299  0.01355114 -0.00891853\n",
      "  0.04934703  0.04002254 -0.0325513   0.0134212   0.09574379  0.04485405\n",
      "  0.02175673 -0.00770148  0.00949134  0.04822486  0.02594588  0.05673572\n",
      "  0.08999248 -0.04783185]\n",
      "[ 0.00295674 -0.01976902  0.00968274  0.08552385  0.18202852  0.20731147\n",
      "  0.19420313 -0.00236286  0.17333006  0.05908928 -0.04687618 -0.01397707\n",
      "  0.06040296  0.04170014 -0.00105445 -0.03799755  0.1819234   0.07885158\n",
      "  0.02389838  0.22803031]\n",
      "0.11414828328726837\n",
      "-0.3291196405243532\n",
      "n_round = 90, n_count = 0\n",
      "[-0.03212262 -0.09263047  0.02348166 -0.03357385 -0.03923798 -0.04469812\n",
      " -0.00676136 -0.00877255  0.02505486  0.0183422  -0.04035392 -0.00793001\n",
      "  0.03606498 -0.04558532 -0.01048169 -0.02670872 -0.00545664 -0.00395858\n",
      " -0.03172341  0.01255262]\n",
      "[ 0.12295444  0.14434285  0.05415558  0.109469    0.00651787  0.05967724\n",
      "  0.05246491  0.12155519 -0.01584561  0.02263314  0.12032273  0.1216197\n",
      "  0.06903014  0.02256824  0.1115704  -0.00247069  0.10628059  0.1125561\n",
      "  0.07524953  0.01231598]\n",
      "-0.03081321925301278\n",
      "-0.19357163887722206\n",
      "n_round = 90, n_count = 5000\n",
      "[ 0.01931727  0.00718494  0.00944789 -0.01619167  0.01135611  0.02002519\n",
      "  0.01507126  0.01887653  0.01725567  0.02805299  0.01575766 -0.01384849\n",
      "  0.01658924 -0.00358897  0.02316547  0.00930663  0.00861904  0.03879204\n",
      "  0.02937531  0.00858508]\n",
      "[-0.01324354  0.07481303  0.03648902  0.13890103  0.1001962   0.0140343\n",
      "  0.04379647  0.12449921  0.12309539 -0.00576525  0.05191835  0.08626366\n",
      "  0.05557302  0.09273032  0.00033644  0.08830358  0.04407854 -0.00294915\n",
      "  0.06083682  0.06041444]\n",
      "-0.02463155424044551\n",
      "-0.22099363951079018\n",
      "n_round = 90, n_count = 10000\n",
      "[ 0.0087916   0.00181276 -0.01056715 -0.00078426  0.00020471 -0.00210409\n",
      " -0.00126559 -0.0061478   0.0052093  -0.01105858 -0.00831126 -0.02329722\n",
      " -0.00071336 -0.00961844 -0.01392496  0.00430171 -0.00229185 -0.01791968\n",
      " -0.01565717  0.00525837]\n",
      "[ 0.25191102  0.22272657  0.11150427  0.19146102  0.14090056  0.18385198\n",
      "  0.21685203  0.08077242  0.13376622 -0.00435522  0.15289128  0.0411046\n",
      "  0.1947784   0.10334589  0.12159171  0.17047323  0.14015933  0.03694277\n",
      "  0.06137754  0.09245142]\n",
      "-0.013173884008689674\n",
      "0.6512112196817389\n",
      "n_round = 90, n_count = 15000\n",
      "[ 0.08531684  0.06984605  0.03955496  0.06059689  0.00923215 -0.01413815\n",
      "  0.04399627  0.03303262 -0.03125906  0.01086748  0.08721854  0.04035879\n",
      "  0.01974701 -0.01329215  0.00739778  0.04079804  0.0169875   0.05516016\n",
      "  0.08768219 -0.04219748]\n",
      "[ 0.00069773 -0.02037287  0.00813881  0.07899307  0.17112326  0.19547614\n",
      "  0.18175004 -0.00308174  0.16395327  0.05535057 -0.0462957  -0.01416632\n",
      "  0.05638259  0.03952218 -0.00118956 -0.0368214   0.17079293  0.07291428\n",
      "  0.02039353  0.21576107]\n",
      "0.12030561066739032\n",
      "-0.3335249879476592\n",
      "SVD trained\n",
      "生成训练数据...\n",
      "\n",
      "train.csv:500 (userId, eventId)=(123290209, 1887085024)\n",
      "train.csv:1000 (userId, eventId)=(272886293, 199858305)\n",
      "train.csv:1500 (userId, eventId)=(395305791, 1582270949)\n",
      "train.csv:2000 (userId, eventId)=(527523423, 3272728211)\n",
      "train.csv:2500 (userId, eventId)=(651258472, 792632006)\n",
      "train.csv:3000 (userId, eventId)=(811791433, 524756826)\n",
      "train.csv:3500 (userId, eventId)=(985547042, 1269035551)\n",
      "train.csv:4000 (userId, eventId)=(1107615001, 173949238)\n",
      "train.csv:4500 (userId, eventId)=(1236336671, 3849306291)\n",
      "train.csv:5000 (userId, eventId)=(1414301782, 2652356640)\n",
      "train.csv:5500 (userId, eventId)=(1595465532, 955398943)\n",
      "train.csv:6000 (userId, eventId)=(1747091728, 2131379889)\n",
      "train.csv:6500 (userId, eventId)=(1914182220, 955398943)\n",
      "train.csv:7000 (userId, eventId)=(2071842684, 1076364848)\n",
      "train.csv:7500 (userId, eventId)=(2217853337, 3051438735)\n",
      "train.csv:8000 (userId, eventId)=(2338481531, 2525447278)\n",
      "train.csv:8500 (userId, eventId)=(2489551967, 520657921)\n",
      "train.csv:9000 (userId, eventId)=(2650493630, 87962584)\n",
      "train.csv:9500 (userId, eventId)=(2791418962, 4223848259)\n",
      "train.csv:10000 (userId, eventId)=(2903662804, 2791462807)\n",
      "train.csv:10500 (userId, eventId)=(3036141956, 3929507420)\n",
      "train.csv:11000 (userId, eventId)=(3176074542, 3459485614)\n",
      "train.csv:11500 (userId, eventId)=(3285425249, 2271782630)\n",
      "train.csv:12000 (userId, eventId)=(3410667855, 1063772489)\n",
      "train.csv:12500 (userId, eventId)=(3531604778, 2584839423)\n",
      "train.csv:13000 (userId, eventId)=(3686871863, 53495098)\n",
      "train.csv:13500 (userId, eventId)=(3833637800, 2415873572)\n",
      "train.csv:14000 (userId, eventId)=(3944021305, 2096772901)\n",
      "train.csv:14500 (userId, eventId)=(4075466480, 3567240505)\n",
      "train.csv:15000 (userId, eventId)=(4197193550, 1628057176)\n",
      "生成预测数据...\n",
      "\n",
      "test.csv:500 (userId, eventId)=(182290053, 2529072432)\n",
      "test.csv:1000 (userId, eventId)=(433510318, 4244463632)\n",
      "test.csv:1500 (userId, eventId)=(632808865, 2845303452)\n",
      "test.csv:2000 (userId, eventId)=(813611885, 2036538169)\n",
      "test.csv:2500 (userId, eventId)=(1010701404, 303459881)\n",
      "test.csv:3000 (userId, eventId)=(1210932037, 2529072432)\n",
      "test.csv:3500 (userId, eventId)=(1452921099, 2705317682)\n",
      "test.csv:4000 (userId, eventId)=(1623287180, 1626678328)\n",
      "test.csv:4500 (userId, eventId)=(1855201342, 2603032829)\n",
      "test.csv:5000 (userId, eventId)=(2083900381, 2529072432)\n",
      "test.csv:5500 (userId, eventId)=(2318415276, 2509151803)\n",
      "test.csv:6000 (userId, eventId)=(2528161539, 4025975316)\n",
      "test.csv:6500 (userId, eventId)=(2749110768, 4244406355)\n",
      "test.csv:7000 (userId, eventId)=(2927772127, 1532377761)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "test.csv:7500 (userId, eventId)=(3199685636, 1776393554)\n",
      "test.csv:8000 (userId, eventId)=(3393388475, 680270887)\n",
      "test.csv:8500 (userId, eventId)=(3601169721, 154434302)\n",
      "test.csv:9000 (userId, eventId)=(3828963415, 3067222491)\n",
      "test.csv:9500 (userId, eventId)=(4018723397, 2522610844)\n",
      "test.csv:10000 (userId, eventId)=(4180064266, 2658555390)\n"
     ]
    }
   ],
   "source": [
    "RS = RecommonderSystem()\n",
    "print (\"生成训练数据...\\n\")\n",
    "generateRSData(RS,train=True,  header=True)\n",
    "\n",
    "print (\"生成预测数据...\\n\")\n",
    "generateRSData(RS, train=False, header=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "时间、地点等特征都没有处理了，可以考虑用户看到event的时间与event开始时间的差、用户地点和event地点的差异。。。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1、基于用户和活动的协同过滤只是利用k近邻的方法，保存了原本的数据，当测试新数据时进行比对和计算，训练快，测试慢  \n",
    "2、基于模型的协同过滤是训练了模型，然后可以抛弃原始训练数据，训练慢，测试快——线上的系统应该用这个更合理一些  \n",
    "3、SVD分解应用很多  \n",
    "    推荐系统中用到的SVD分解称为：隐因子分解  \n",
    "    PCA降维里面用到SVD分解称为：主成分分析  \n",
    "    语言文本用到SVD分解称为：隐性语义分析  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "将生成的数据（修炼成果）读进来检查下有木有问题"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>invited</th>\n",
       "      <th>userCF_reco</th>\n",
       "      <th>evtCF_reco</th>\n",
       "      <th>svdCF_reco</th>\n",
       "      <th>user_reco</th>\n",
       "      <th>evt_p_reco</th>\n",
       "      <th>evt_c_reco</th>\n",
       "      <th>user_pop</th>\n",
       "      <th>frnd_infl</th>\n",
       "      <th>evt_pop</th>\n",
       "      <th>interested</th>\n",
       "      <th>not_interested</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>[[-3.08693188e-05]]</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.010140</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.001099</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000018</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>[[-1.35525272e-20]]</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.041679</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.001099</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-0.000012</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>[[1.00000882]]</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.516199</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>0.001099</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000021</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>[[0.]]</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.001099</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000041</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>[[0.25293482]]</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.203597</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.001099</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-0.000099</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   invited          userCF_reco  evtCF_reco  svdCF_reco  user_reco  \\\n",
       "0        0  [[-3.08693188e-05]]         0.0    0.010140        0.0   \n",
       "1        0  [[-1.35525272e-20]]         0.0    0.041679        0.0   \n",
       "2        0       [[1.00000882]]         1.0    0.516199        0.0   \n",
       "3        0               [[0.]]         0.0    0.000000        0.0   \n",
       "4        0       [[0.25293482]]         0.0    0.203597        0.0   \n",
       "\n",
       "   evt_p_reco  evt_c_reco  user_pop  frnd_infl   evt_pop  interested  \\\n",
       "0         0.0         0.0  0.001099        0.0  0.000018           0   \n",
       "1         0.0         0.0  0.001099        0.0 -0.000012           0   \n",
       "2        -1.0        -1.0  0.001099        0.0  0.000021           1   \n",
       "3         0.0         0.0  0.001099        0.0  0.000041           0   \n",
       "4         0.0         0.0  0.001099        0.0 -0.000099           0   \n",
       "\n",
       "   not_interested  \n",
       "0               0  \n",
       "1               0  \n",
       "2               0  \n",
       "3               0  \n",
       "4               0  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "new_train_data = pd.read_csv(\"RS_train.csv\")\n",
    "new_train_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>invited</th>\n",
       "      <th>userCF_reco</th>\n",
       "      <th>evtCF_reco</th>\n",
       "      <th>svdCF_reco</th>\n",
       "      <th>user_reco</th>\n",
       "      <th>evt_p_reco</th>\n",
       "      <th>evt_c_reco</th>\n",
       "      <th>user_pop</th>\n",
       "      <th>frnd_infl</th>\n",
       "      <th>evt_pop</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000199</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000051</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000199</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000035</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000199</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000115</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000199</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000025</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000199</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-0.000014</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   invited  userCF_reco  evtCF_reco  svdCF_reco  user_reco  evt_p_reco  \\\n",
       "0        0            0           0         1.0        0.0         0.0   \n",
       "1        0            0           0         1.0        0.0         0.0   \n",
       "2        0            0           0         1.0        0.0         0.0   \n",
       "3        0            0           0         1.0        0.0         0.0   \n",
       "4        0            0           0         1.0        0.0         0.0   \n",
       "\n",
       "   evt_c_reco  user_pop  frnd_infl   evt_pop  \n",
       "0         0.0  0.000199        0.0  0.000051  \n",
       "1         0.0  0.000199        0.0  0.000035  \n",
       "2         0.0  0.000199        0.0  0.000115  \n",
       "3         0.0  0.000199        0.0  0.000025  \n",
       "4         0.0  0.000199        0.0 -0.000014  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_test_data = pd.read_csv(\"RS_test.csv\")\n",
    "new_test_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "发现训训练集和测试集生成的数据还是比较正常的，训练集中的userCF_reco这列数据格式有点问题，显示的是一个二维数组，但值确实只有一个，可能是上面处理的时候没有进行转化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "小结：  \n",
    "1、如果遇到训练时间比较长的作业，跑起来都不知道运行到哪里了，程序是否跑死了，可以通过在各关键计算出添加打印，看计算到哪一步，但打印也不能加的太多，这次的作业之前加了几个没有过滤的打印，打印信息太多，结果这个ipynb文件占了52M，下次打开的时候加载很长时间，可能会卡死打不开  \n",
    "2、本次作业涉及到很多矩阵的计算，可以通过打印方式，保证每一步计算时维数和行列数为预期值，在深度学习中可能也需要这样，深度学习可以使用TensorFlow里封装好的包，但不知道到时是否还需要手撕梯度下降计算（eg.反向传播）   \n",
    "3、计算SVD分解是，参数的初始化Puk、Qki、Bu、Bi是随机给出的或直接是零值，不知道这个几个参数的初始化内容是否影响最后的训练模型，猜测应该最终也是得到局部最优解，而不同的初始化可能得到的局部最优解不同    \n",
    "4、计算多个数据时最好还是封装能计算整个矩阵的函数，单个数值计算的速度确实慢好多，这要求对python的几种数据结构和numpy库有更多的理解，这点我还需要改善  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
